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System Review Report 
August 3, 2020 


The Honorable Mark L. Greenblatt 
Inspector General 

U.S. Department of the Interior 
1849 C Street NW 

Washington, D.C. 20240 


Dear Mr. Greenblatt: 


We have reviewed the system of quality control for the audit organization of the U.S. 
Department of the Interior (DOI), Office of the Inspector General (OIG), in effect for the year ended 
September 30, 2019. A system of quality control encompasses DOI OIG’s organizational structure, 
and the policies adopted, and procedures established to provide it with reasonable assurance of 
conforming with Government Auditing Standards (GAS). The elements of quality control are 
described in those standards. 


DOI OIG is responsible for establishing and maintaining a system of quality control that is 
designed to provide it with reasonable assurance that the organization and its personnel comply 
with professional standards and applicable legal and regulatory requirements in all material 
respects. Our responsibility is to express an opinion on the design of the system of quality control 
and DOI OIG's compliance therewith, based on our review. 


Our review was conducted in accordance with GAS and the Council of the Inspectors General 
on Integrity and Efficiency (CIGIE) Guide for Conducting Peer Reviews of the Audit Organizations of 
Federal Offices of Inspector General. During our review, we interviewed DOI OIG personnel and 
obtained an understanding of the nature of the DOI OIG audit organization and the design of DOI 
OIG's system of quality control sufficient to assess the risks implicit in its audit function. 


Based on our assessments, we selected audits and administrative files to test for conformity 
with professional standards and compliance with DOI OIG’s system of quality control. The audits 
selected represented a reasonable cross-section of the DOI OIG's audit organization. Prior to 
concluding the peer review, we reassessed the adequacy of the scope of the peer review procedures 
and met with DOI OIG management to discuss the results of our review. We believe that the 
procedures we performed provide a reasonable basis for our opinion. 


In performing our review, we obtained an understanding of the system of quality control for 
the DOI OIG audit organization. In addition, we tested compliance with DOI OIG's quality control 
policies and procedures to the extent we considered appropriate. These tests covered the 
application of DOI OIG's policies and procedures on selected audits. Our review was based on 
selected tests; therefore, the review would not necessarily detect all weaknesses in the system of 
quality control or all instances of noncompliance with it. 


There are inherent limitations in the effectiveness of any system of quality control; therefore, 
noncompliance with the system of quality control may occur and not be detected. Projection of any 
evaluation of a system of quality control to future periods is subject to the risk that the system of 
quality control may become inadequate because of changes in conditions, or because the degree of 
compliance with the policies or procedures may deteriorate. 


We noted the following deficiency during our review: 


The DOI OIG’s implementation of its system of quality control did not provide 
reasonable assurance that two of the six audits reviewed were performed in 
compliance with GAS and DOI OIG policies and procedures. 


We determined that the DOI OIG's control structure did not assure compliance with GAS and 
its own policies and procedures for sampled audits 2017-FIN-053 and 2018-FIN-032, both issued by 
the same DOI OIG Unit. We determined that DOI OIG auditors did not use sufficient and 
appropriate evidence to support some statements and conclusions. This can be attributed to 
breakdowns with audit planning, assessment of evidence, supervision, and compliance with quality 
control policies and procedures. After discussions with DOI OIG during our review, DOI OIG 
withdrew these two reports from its public website. DOI OIG officials stated that it is performing 
additional audit work and Quality Assurance Reviews (QAR) for these reports, and, once complete, 
these reports may be reissued and re-posted to the DOI OIG public.' Additionally, DOI OIG officials 
told us that QARs will be completed for the rest of the audits issued by the this Unit over the course 
of our review period. 


Enclosure 1 provides a detailed, technical discussion of the issues related to the deficiency 
we identified and our recommendations for corrective action. Enclosure 2 to this report identifies 
the DOI OIG offices that we visited, a description of the work we performed, and a listing of the 
audits we reviewed. Finally, the DOI OIG response is included as Enclosure 3. 


In our opinion, except for the deficiency described above and in the enclosure, the system of 
quality control in effect for the DOI OIG's audit organization for the year ending September 30, 2019, 
was suitably designed and complied with to provide the DOI OIG with reasonable assurance of 
performing and reporting in conformity with applicable professional standards in all material 
respects. Audit organizations can receive a rating of pass, pass with deficiencies, or fail. The DOI 
OIG has received an External Peer Review rating of pass with deficiency. 


1 DOI OIG reissued one of these reports on April 28, 2020. We did not review this reissued report for 
GAS compliance. 


In addition to reviewing the DOI OIG's system of quality control to assure adherence with 
GAS, we also applied certain limited procedures in accordance with CIGIE guidance. The applicable 
CIGIE guidance involved DOI OIG monitoring of engagements performed by Independent Public 
Accountants (IPA) under contract, where the IPA served as the principal auditor. 


Please note that the monitoring of engagements performed by IPAs is not an audit and, 
therefore, is not subject to the requirements of GAS. The purpose of our limited procedures was to 
determine whether the DOI OIG has controls to ensure that the IPAs performed contracted work in 
accordance with professional standards. Our objective was not to express an opinion, and we do 
not express an opinion on the DOI OIG monitoring the work performed by IPAs. 


Thank you again for the cooperation and courtesies extended to our staff during the review. 
Sincerely, 
Michael E. Horowitz 


Inspector General 


Enclosures 


Enclosure 1 


Deficiency: DOI OIG’s Implementation of its System of Quality Control Did Not Provide Reasonable 
Assurance that Two of the Six Audits Reviewed Were Performed in Compliance with Government 
Auditing Standards 


We reviewed the adequacy of the Department of the Interior (DOI) Office of Inspector 
General's (OIG) policies and procedures to determine whether those policies and procedures, if 
properly adopted and implemented, would provide the DOI OIG with reasonable assurance of 
compliance with Government Auditing Standards (GAS). The Audits, Inspections and Evaluations 
(AIE) Policy Manual stated that the internal quality control system consists of: (1) supervision of 
work performed, (2) the referencing of individual reports, and (3) completion of Quality Assurance 
(QA) checklists.* Generally, our review of the policies and procedures concluded that they were 
adequate if properly implemented. 


However, we found that two out of the six performance audits, both issued by the same DOI 
OIG Unit, we reviewed did not fully comply with GAS or DOI OIG policies and procedures.? 


The first was an audit of three National Park Service (NPS) contracts issued in support of the 
Grand Canyon River Logistics Program (NPS audit). According to DOI OIG, this audit was initiated 
after it received allegations of certain contracting irregularities. The audit objectives were to 
determine whether the NPS: (1) complied with the Federal Acquisition Regulation (FAR) pre-award 
requirements; (2) awarded the contracts’ task orders in accordance with the terms of the contracts 
and FAR subpart 16.5; and (3) monitored and oversaw the contracts during their period of 
performance, including monitoring contractor compliance with drug screening and criminal 
background checks. The report contained no recommendations and concluded that NPS complied 
with applicable federal regulations and contract provisions during the contracts’ pre-award periods 
and when awarding the 13 task orders, and that the NPS monitored contractors’ compliance with 
drug screening and background check policies. Further, DOI OIG stated that it did not identify any 
evidence that supported the allegations it received. 


The second was an audit of costs claimed by the Chicago Horticultural Society (CHS) under a 
grant awarded by the Bureau of Land Management (BLM) (CHS audit).°> The audit objective was to 
determine whether the interim costs claimed by the CHS between May 1, 2015, and August 31, 2017, 
were allowable and allocable, and in compliance with applicable federal regulations, BLM policies 
and procedures, and contract terms and conditions. The DOI OIG made 10 recommendations to the 
BLM and identified $530,537 in questioned costs. 


2 Through our review, we found that DOI OIG revised and restructured its policies effective July 1, 
2019. However, none of the audits within our sample were conducted under these new policies. Therefore, 
our review focused on the AIE Policy Manual effective through June 30, 2019. 


3 For more information regarding our scope and methodology, see Enclosure 2. 


4 DOI OIG Audit Report 2018-FIN-032: The NPS Complied With the Terms of Three Contracts for the 
Grand Canyon River Logistics Program. 


> DOI OIG Audit Report 2077-FIN-053. The Chicago Horticultural Society Should Improve Its Financial 
Management System to Receive Federal Funds. 


In these two reports, we identified noncompliance with reporting standards that resulted in 
some report statements and conclusions for which the audit file lacked sufficient and appropriate 
supporting evidence. Specifically, in our review of the NPS report, we found statements and 
conclusions that were inaccurate or not supported by sufficient evidence and found analyses that 
were not documented or were incomplete. In our review of the CHS audit report, we were unable to 
reach the same reported questioned costs as the CHS team because the evidence in the audit file 
was insufficient and included an incomplete evaluation of the evidence obtained. 


We also identified several instances where the reported scope and methodologies for both 
reports lacked sufficient and appropriate supporting evidence. For the CHS audit report, we 
determined that the DOI OIG description of the scope used to address the audit objectives was 
incomplete. For the NPS audit, the methodology included inaccurate descriptions of the testing 
conducted and the omission of relevant criteria used to support the report's conclusions. 


Audit Planning 


GAS requires auditors to assess the risk of the possibility that the auditors’ findings, 
conclusions, recommendations, or assurance may be improper or incomplete, referred to as “audit 
risk.” For the NPS audit, the audit team identified the risk to the audit as high, however the audit 
team did not implement steps to mitigate this risk, as required by GAS and DOI OIG policy. Further, 
we found that the audit team did not adequately develop its assessment of audit risk during 
planning, specifically in the areas of: nature of the program, internal controls, information systems 
controls, laws and regulations, criteria, prior corrective actions, and evidence. For example, it 
appeared the risk assessment of internal controls relied solely on interviews with NPS personnel, 
which we found to be insufficient. We also found no evidence that the audit team evaluated the 
NPS's prior corrective actions taken to address the findings of a previous DOI OIG investigative 
report. 


In addition, given the allegations of contracting irregularities associated with the NPS audit, 
the audit team acknowledged the potential for fraud related to the audit objectives. However, we 
found that the audit file did not include, as required by GAS, procedures for the DOI OIG to obtain 
reasonable assurance of detecting fraud, determine whether fraud had likely occurred, and express 
its effect on audit findings. 


Evidence and Documentation 





In both the NPS and CHS audit, we found weaknesses with DOI OIG’s implementation of GAS 
requirements and DOI OIG policies related to the sufficiency and appropriateness of evidence used 
to support the auditor's findings and conclusions in relation to the audit objectives. 


The objectives and scope of the NPS audit indicated that the audit team would review three 
contracts and verify that task orders were awarded in compliance with contract terms and 
conditions. However, we found that the audit team relied heavily on the contract award's 
solicitation, rather than the contracts, for its analysis and conclusions for two of the three contracts. 
We believe this is insufficient because solicitations are not the same as executed contracts and do 
not contain contract requirements. 


Further, in its audit file and as stated in the final audit report, the NPS audit team concluded 
that NPS issued 13 task orders in compliance with FAR. However, in our review of the audit file, we 
did not find any methodology or testing on which the audit team based this determination. In 
another example, the audit team concluded that the NPS ensured there was adequate competition 
amongst contractors; however, we found insufficient evidence in the audit file to support that 
conclusion. 


The NPS audit team also did not assess the reliability of evidence provided by the audited 
entity to support corrective actions taken to address the audit team’s determination that NPS 
employees were performing required contractor tasks. Instead, the DOI OIG accepted an undated 
and unsigned electronic document as sufficient evidence to demonstrate that NPS clarified tasks for 
its employees and contractors. However, this document lacked certain identifiers that would 
demonstrate that it was an official document or information as to who sent or received it. Further, 
we found evidence in the audit file in which one of the contractors expressed disagreement with the 
NPS document. However, nothing in the audit file indicated that the audit team considered this 
information in concluding on this matter. 


In addition, the audit team concluded that NPS contracting personnel never compared the 
government cost estimates to the bid amounts submitted by potential vendors. However, the NPS 
audit team referenced testimonial evidence that contradicted this conclusion, and we found no 
other testing or evidence on this matter in the audit file. Similarly, while the audit team concluded 
that contractors did not receive unequal access to information and that NPS employees performed 
required contractor tasks, we found inadequate support documented in the audit file for these 
conclusions. 


For the CHS audit, the CHS audit team concluded that CHS incorrectly claimed $189,902 for 
intern expenses. However, we found that the audit team calculated questioned costs using an 
estimated hourly rate rather than using the available accounting records, which included the actual 
amounts CHS claimed for intern salary, fringe benefits, and indirect costs. As a result, in its report 
the DOI OIG misstated questioned costs for salary, fringe benefits, and indirect costs. In addition, 
the CHS audit team found that seven interns hired by CHS exceeded the allowed age limit. However, 
we determined that the audit team relied on a CHS spreadsheet and performed no additional 
testing to validate the reliability of intern ages provided on the spreadsheet. Further, there were 
two additional instances where the CHS audit team questioned costs, but we, as experienced 
auditors and having no connection to this audit, were unable to reach the same conclusion based on 
our review of the methodology described and evidence in the audit file. 





Supervision and Quality Control 


We found indications that both audits lacked adequate supervision and quality control 
according to both GAS and DOI OIG policies. 


Specifically, in the NPS audit, we identified supervisory weaknesses with ensuring: (1) the 
sufficiency and appropriateness of the evidence obtained to support findings and conclusions; 
(2) the auditors' work was accurate, complete, and clear to enable an experienced auditor having no 
previous connection to the audit, to understand the nature, timing, extent, and results of 
procedures performed, the evidence obtained and its source, and the conclusions reached; (3) the 


QA checklist was complete and accurate; and (4) the views of responsible auditee officials were 
reported. 


Similarly, in the CHS audit file, we identified: (1) undocumented significant changes made to 
the audit objectives and scope, as well as an incomplete internal control assessment; (2) insufficient 
and inappropriate evidence used to support findings and conclusions and insufficient detail of the 
audit documentation; (3) the report's omission of the views of responsible auditee officials and 
corrective actions taken for an identified finding; (4) and the inclusion of a report recommendation 
that did not address the deficiency identified. 


Finally, we identified issues with both of the audit teams’ implementation of quality control 
requirements. As mentioned above, we found that both audit teams relied on incomplete and 
insufficient evidence to support the report findings and conclusions and that this was not identified 
and corrected through adequate supervision. Further, in respect to the other aspects of the DOI 
OIG quality control system, we found that both audit teams: (1) inadequately cross-indexed the 
reports, which was evidenced by unindexed report facts, insufficient indexes to support the reports’ 
findings and conclusions, and inadequate evidence, none of which was identified or corrected 
through the independent referencing process; and (2) did not document complete and accurate QA 
checklists. 


Conclusion and Recommendation 


We believe the noncompliance with GAS requirements in both the NPS and CHS reports can 
be attributed to the inadequate implementation of these standards and DOI OIG policies in the 
areas of audit planning, assessment of evidence, supervision, and quality control. Overall, if the NPS 
audit team had more fully developed their audit procedures to better obtain reasonable assurance 
that the evidence obtained was sufficient and appropriate to support the auditors’ findings and 
conclusions in relation to the audit objectives, some of the deficiencies we identified may have been 
avoided. Further, based on our review of the NPS and CHS audit files, we believe these files were 
not prepared with sufficient enough detail to enable an experienced auditor having no previous 
connection to the audit, to understand the nature, timing, extent, and results of procedures 
performed; the evidence obtained and its source; and the conclusions reached, including evidence 
that supports the auditors’ significant judgments and conclusions. We believe more robust 
supervision of these audit teams and proper implementation of the DOI OIG quality assurance 
process could have also helped prevent the deficiencies we identified. 


Based on our review, we provided the DOI OIG the following recommenadation. Enclosure 3 of this 
report includes the DOI OIG’s response, in which the DOI OIG concurs with this recommendation 
and states the corrective actions it has or will be taking to address the recommendation, including 
updates to DOI OIG policies and standard operating procedures, Quality Assurance Reviews of the 
two reports discussed in this report as well as other reports issued or to be issued by the same 
division, and the training of staff. 


Recommendation 1 - To ensure compliance with GAS requirements, assess its current policies and 
procedures related to audit planning, documentation of evidence, supervision, and reporting to 
obtain a better understanding of the cause for the deficiency identified in this review, and update 
and train staff on applicable policies and procedures as necessary. 


Enclosure 2 
Scope and Methodology 


We tested compliance with the DOI OIG audit organization's system of quality control to the 
extent we considered appropriate. These tests included a review of 7 of 22 audit reports issued 
during the period October 1, 2018, through September 30, 2019. Additionally, we selected one audit 
report issued in Fiscal Year (FY) 2018 in order to review an audit report that had undergone the DOI 
OIG's Quality Assurance Review (QAR). We also reviewed the internal quality control reviews 
performed by DOI OIG. 


Within the seven audits issued during FY 2019, two are reports where the DOI OIG’ monitored 
the work performed by an Independent Public Accounting Firm (IPA) where the IPA served as the 
auditor during the period October 1, 2018, through September 30, 2019. During this period, the DOI 
OIG contracted for the audits of its agency's financial statements and its agency's information 
security programs that were performed in accordance with Government Auditing Standards. 


We visited DOI OIG offices in Washington, D.C.; Herndon, Virginia; and Lakewood, Colorado. 
In addition, we reviewed the training records of selected DOI OIG employees to determine 
whether they had obtained the required continuing professional education credits. We determined 


that the training data we received from the DOI OIG and used in this report were sufficiently reliable 
given the audit objectives and intended use of the data. 


Table 1. Reviewed Audits Issued by DOI OIG 





Report Date Report Number Report Title 
WSFR Grants Awarded to the State of California, Department of 
4/15/2019 2017-WR-064 Fish and Wildlife, From July 1, 2014, Through June 30, 2016 
WSFR Grants Awarded to the State of South Dakota, Department of 
3/29/2019 2018-CR-001 Game, Fish and Parks, From July 1, 2015, Through June 30, 2017 
Improvement Needed in Internal Controls for the Use of 
3/26/2019 2017-ER-015-A Convenience Checks at DOI 
Independent Auditors' Performance Audit Report on DOI Federal 
3/13/2019 2018-ITA-043 Information Security Modernization Act for Fiscal Year 2018 
The Chicago Horticultural Society Should Improve Its Financial 
3/13/2019 2017-FIN-053 Management System to Receive Federal Funds 
Independent Auditors' Report on DOI Financial Statements for 
11/15/2018 2018-FIN-037 Fiscal Years 2018 and 2017 
The NPS Complied With the Terms of Three Contracts for the 
10/30/2018 2018-FIN-032 Grand Canyon River Logistics Program 


Audit of Grant No. P13AF00113 Between the National Park Service 
and the Connecticut Department of Economic and Community 
1/8/2018 2017-FIN-032 Development 
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JUL 23 2020 


Jason Malmstrom 

Assistant Inspector General for Audit 
U.S. Department of Justice 

Office of Inspector General 

950 Pennsylvania Avenue, NW 
Washington, DC 20530 


Subject: Response to Draft System Review Report 
Dear Mr. Malmstrom: 


Thank you for the opportunity to comment on the draft results of your peer review of our 
office’s audit operations. Your report made the following recommendation: 


To ensure compliance with GAS requirements, assess its current policies and procedures 
related to audit planning, documentation of evidence, supervision, and reporting to 
obtain a better understanding of the cause for the deficiency identified in this review, and 
update and train staff on applicable policies and procedures as necessary. 


We concur with your recommendation and have implemented the following corrective 
actions. 


e Initiated our standard operating procedure (SOP) project in May 2020. Initial SOPs 
include Assignment Documentation, Assignment Planning, Audit Risk, Fraud, 
Indexing and Referencing, and Reporting Process. 


e Scheduled training on supervisory review of working papers in TeamMate+ for all 
supervisors and managers in September 2020. 


e Updated our policy for obtaining and reporting the views of responsible auditee 
officials for audit reports with no findings. 


e Initiated an in-depth review of the AIE division responsible for most of your 
identified deficiencies. To date we have: 


o Completed an internal quality assurance review (QAR) of The Chicago 
Horticultural Society Should Improve Its Financial Management System to 
Receive Federal Funds audit (“CHS audit”) and reposted the report with minor 
modifications. 


Office of Inspector General | Washington, DC 


Enclosure 3 


o Started a QAR of The NPS Complied With the Terms of Three Contracts for the 
Grand Canyon River Logistics Program audit (“NPS audit”) and plan to repost 
that report in July. 


o Initiated QARs of the five other audits issued from October 1, 2018, through 
September 30, 2019, and all new audits issued from October 1, 2019, through 
December 31, 2020, by the same division. 


o Completed an after action review (AAR) of the CHS audit in June 2020 and plan 
to perform an AAR of the NPS audit. 


We also plan to provide concentrated indexing and referencing training for all AIE staff 
in the division. 


Although we concur with your recommendation, we continue to disagree with your 
conclusion that the CHS report lacked sufficient and appropriate evidence to support some 
statements and conclusions. We pulled the CHS report from the public OIG website and 
completed a QAR of the CHS audit file. We determined that the audit team obtained sufficient 
and appropriate evidence to support all statements and conclusions in the report. We found, 
however, that the audit team’s analysis was not complete, which resulted in incorrect statements 
that required correction in the report. 


We are committed to producing high-quality audits. We appreciate the professionalism 
your staff displayed during the review. 


If you have any questions, please contact me at 202-208-5745. 
Sincerely, 


Kimberly E Af Gin 


Kimberly E. McGovern 
Assistant Inspector General for Audits, 
Inspections and Evaluations 
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